1. Technical Field
The present invention relates generally to an apparatus and method for designing a display for user interaction and, more particularly, to an apparatus and method for designing a display for three-dimensional (3D) user interaction in a near-body space.
2. Description of the Related Art
A procedure in which human beings visually recognize an object as a three-dimensional (3D) stereoscopic image is influenced by a variety of visual factors.
Since human beings have two eyes on left and right sides of their faces, binocular disparity information occurs between images of external world that are observed, and such images are recognized as a single 3D stereoscopic image by the brain. The principle of the recognition of a 3D effect felt by such binocular disparity has been applied to popularized 3D stereoscopic display devices. A display device outputs images to be applied to both eyes of a user. The user may view 3D stereoscopic image content by wearing a device (for example, 3D stereoscopic eyeglasses) for separating left/right images corresponding to respective eyes.
As wearable displays, there are introduced a Head Mounted Display (HMD), a face Mounted Display (FMD), a Head-Up Display (HUD), a Near-Eye Display (NED), an Eye Glass Display (EGD), etc.
Such a wearable display denotes a device for presenting virtual information created by a computer to the organ of sight of a user. Wearable displays may be chiefly classified into see-closed schemes for separating the visual part of the user from an external space and see-through schemes for allowing the visual part and the external space to be simultaneously viewed.
Here, the see-through schemes are classified into low-class categories such as an optical see-through scheme for viewing an external space through transmissive/reflective optical modules and a vision-based see-through scheme for processing information via an image acquisition device such as a camera and presenting processed information to the user.
In the technical fields of virtual reality, augmented reality, and mixed reality, when experience in virtual content is transferred to the user, a wearable display has been utilized as a representative interface for presenting personalized immersive content.
Around the year 2010, with the popularization of Hollywood movies and home appliance markets for 3D TVs to which 3D visualization technology was applied, general consumer interest in 3D stereoscopic images has increased. However, it is impossible to completely (100%) reconstruct a natural phenomenon based on the visual recognition of a 3D stereoscopic space due to technical limitations. Reports on side effects from the utilization of related technology have increased, and research into the solution of related problems based on human factor-related issues has been conducted in the fields of research and industry.
Holographic display technology designated as an ideal 3D stereoscopic image visualization technology implements a procedure in which a light source or light from the light source is reflected and scattered from the surface of an object and is perceived by human eyes, as in the case of a natural environment, but there are still many limitations in the implementation of a commercialization level quality and the mass production of actual products.
Most of 3D stereoscopic image displays that may be currently commercialized represent a sense of depth of an object based on binocular disparity information. Even an auto-stereoscopic display (or glass-free display) which does not require the wearing of glasses basically represents a 3D effect using the principle of presenting binocular disparity images to both eyes.
A procedure in which a human being perceives a 3D stereoscopic image is influenced by a plurality of factors, but most 3D commercialized display devices are configured to merely realize a binocular disparity principle. However, a stereoscopic image does not reflect various parameters related to visualization such as the user's viewpoint information, thus representing an inaccurate 3D effect as a primary problem.
Since there is a fundamental problem such as a convergence-accommodation conflict, side effects may be applied to viewers upon representing images in which 3D stereoscopic objects are projected or retracted on a screen, by using stereoscopic image output technology based on binocular disparity.
The present applicant recognized that an optimal visualization space corresponding to a limited area (enabling safe viewing of users while minimizing side effects) is present, via research conducted over several years. However, continuous experience in most cases in which the latest research results are not desirably reflected/optimized causes human factor-related side effects. FIG. 1 is a diagram showing a convergence-accommodation conflict (VA conflict). Parallax denotes a difference between the horizontal positions of an image for a left eye 1 and an image for a right eye 1 projected on an image screen 3. FIG. 1 illustrates the case of positive parallax in which images are separated to allow a virtual 3D object 4 to be located behind the image screen 3, wherein a vergence distance and a focal distance must be identical to each other in a situation in which a human being normally perceives a sense of 3D distance. However, when positive or negative parallax is implemented, such characteristics cannot be satisfied.
Binocular disparity is a factor having a significant effect on the perception of a sense of distance of an object located in a near-body space (refers to a nearby reachable space which an observer can reach by moving his or her legs and arms without moving his or her location, and typically denotes a space within a radius of about 1 m to 1.5 m) among factors required by a human being to perceive a sense of a 3D stereoscopic space. Further, an existing binocular type HMD has a focal distance at which most images of objects are formed within a distance of 2 m to 3 m (an example of a promotional phrase of related products, “If you wear “OOO product,” you can have the feeling of viewing a OO-inch large screen installed at a distance of 2 to 3 m”).
Examples in which a 3D object (for example, a 3D User Interface: UI) around a user is manipulated using a glass-type or a wearable display are frequently produced in movies and product advertising. However, in order to directly interact (such as a touch) with objects within the range of a near-body space, a sense of positioning a 3D object at the fingertip of the user must be transferred, as shown in FIGS. 2A to 2D.
Because most 3D UIs introduced in movies, Commercial Film (CF), etc. are conceptual scenes in which images are combined and represented from the viewpoint of the third person, a user actually encountering the function cannot experience such a natural image. Therefore, there is required technology for visualizing images according to a sense of exact depth within a space defined by a distance of 1 to 2 m around a user's viewpoint via a wearable display.
However, an optical see-through-type wearable display is configured such that an optical module outputs an image at a location spaced at a predetermined distance in a space, and such that the image is always first viewed by the user's eyes. Accordingly, since an unnatural phenomenon occurs in which an image is always viewed to overlap with a virtual image in front of an actual object located closer to the eyes than the virtual image, technology for exactly representing a sense of relative depth must be applied to a wearable display.
At an early stage in which 3D visualization technology is popularized, consumers experiencing 3D stereoscopic images based on the principle of binocular disparity recognize that images providing a new and strange sense are present while viewing unfamiliar images glimmering in front of the eyes, and understand those images to be 3D stereoscopic images. However, consumers realize that this technology makes it impossible to view images for a long period of time due to the above-described fundamental limitations (CA conflict problem), and this impossibility becomes a principal factor that is an obstacle to the popularization of 3D image technology. Therefore, a new idea of an optimization method for solving this problem is required.
In order to naturally support direct interaction in a near-body space for a long period of time, principal parameters for an optical module design to represent binocular stereoscopic images corresponding to the range of a distance of the near-body space are separately present. In the past, a visualization optical module was arbitrarily designed and produced at any position depending on the condition of a manufacturer (for example, guidance for the purpose of product advertising, or guidance for allowing a viewer to view a TV while keeping a typical safe viewing distance of 2 to 3 m or longer) without having to know a specific value. Therefore, criteria for a conventional optical module design are unsuitable for 3D User Interface (UI)/User Experience (UX) conditions requiring direct interaction in the near-body space. In other words, expectation for the utilization of a wearable display presenting an individual-centered 3D stereoscopic visualization space is very high, but there are problems in that interaction in the near-body space, which was realized via the wearable display, was unnatural.
The present applicant has developed virtual/augmented/mixed reality content for a near-body space by utilizing a plurality of existing commercial HMD and FMD, but he or she has undergone difficulty in realizing a natural 3D effect and has obtained an idea of the present invention while conducting research.
As related preceding technology, Korean Patent Application Publication No. 10-2008-0010502 (entitled “Face-mounted display device for mixed reality environment”) discloses technology for matching pieces of image information generated from an actual image and a plurality of artificial images with a single 3D virtual space, and transferring matched information to a user.
The invention disclosed in Korean Patent Application Publication No. 10-2008-0010502 presents a structure capable of combining an external image with information stored in an internal imaging device via a see-through function, and then solves conventional disadvantages occurring when a user independently uses the imaging device.
However, the invention disclosed in the above-described Korean Patent Application Publication No. 10-2008-0010502 merely presents a basic design and a simple application for combining and presenting multiple stereoscopic images on an eyeglass display (EGD) for an Expanded 3D (E3D) platform.